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(54) rule: VIRTUAL FLUOROSCOPIC SYSTEM AND METHOD 

fO 

(57) Abstract: A method and system for obtaining desired subsurface views of a patient target site are disclosed. The system in- 
eludes a scan-data tile for storing, in digital form, volumetric scan data of a surgical site of a patient, a pointer for indicating the 
position and orientation of desired views, and a display screen for placement near the pylient target region. A computational device 

^ operatively connected to the data file, pointer, and display screen functions to reconstruct a 2-D shadowgraph images of the patient 
region from the position and orientation of the pointer. By moving the ix)inter to selected positions and orientations, the user can 

Q view optimal and desired shadowgraph images of the patient in real lime. Also disclosed is a fast -mapping algorithm for mapping 

^ volumetric data onto a two-dimensional screen, and a method and system for registering volumetric scan data of a patient surgical 

^ site with actual patient position. 
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VIRTUAL FLUOROSCOPIC SYSTEM AND METHOD 

Field of the Invention 
5 This invention relates to image-guided surgery, and in particular, to 

registration and fluoroscopic imaging methods. 

Background of the Invention 

In a typical image-guided surgical procedure, pre-operative scan data, 

10 e.g., CT or MRI scan data, of a patient region is obtained. This is done 

conventionally by immobilizing a patient during a scan with a CT or MRI Imaging 
machine. The patient Is then moved to a surgical setting for treatment of the 
target region. During the surgical procedure, the scan data is used to 
reconstruct hidden or subsurface images of the patient target region, to guide or 

15 assist the surgeon in the surgical procedure. For example, the scan data may 
be used to reconstruct a subsurface image of the target region as seen from the 
position and orientation of a surgical instrument being held by the surgeon. 

It is important, of course, for the coordinates of the scan-data images 
being shown to the surgeon to closely match the coordinates of the actual target 

20 region. For this reason, it is important to calibrate the scan data with the actual 
position of the patient, keeping in mind that the scan data is obtained in one 
location, and the patient is moved to another location for the surgical procedure. 
Where the target region is a patient's head region, this coordinate matching 
may be done by placing fixed-position fiducials on the patient's head during the 

25 scanning procedure, and retaining the fiducials at the same head positions for 
the surgical operation. Coordinate matching can then be done by aligning the 
positions of the fiducials In both reference frames. This approach relies on the 
fact that the fiducials placed on rigid, non-deformable structure (e.g., skull) are 
retained in substantially the same position (patient coordinates) for both the 

30 scanning and surgical procedures. 

For other surgical regions, e.g., the spine, it is virtually impossible to 
retain fiducials at fixed patient coordinates between the scanning and surgical 
operations. In this case, it is necessary to recalibrate the positions of the 

1 
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fiducials in the patient coordinate system every time the patient moves or is 
moved. Ideally, the surgeon would want to forego the use of fiducials altogether, 
since their placement involves additional inconvenience to the patient and 
surgeon. 

5 It would therefore be useful to provide an improved system for registering 

pre-op scan data in a patient coordinate system during a surgical operation, to 
be able to use the scan data to accurately construct subsurface image data 
corresponding to the actual patient coordinate system. 

Also during a surgical operation, a surgeon will often wish to check the 

1 0 exact placement of surgical cut, or placement of a tool or implant at the patient 
site. This can be done conventionally, by taking a fluoroscopic image of the 
patient region, and displaying the 2-D image to the surgeon. Unfortunately, the 
surgeon will be limited in the number or times and duration of fluoroscopic 
imagjng, due to the need to limit the amount of x-ray exposure to both the 

15 patient and medical personnel. 

It would therefore be desirable to provide a method and system of virtual 
fluoroscopy that allows a surgeon to view a patient region "fluoroscopically" from 
a number of different angles, and over extended periods of view, without actually 
exposing the patient and others in the surgical theatre to exposure to x-rays. . 

20 In particular, it is would be useful to constmct virtual fluoroscopic images 

in real time, e.g., without significant computation time. 

Summary of the Invention 

Fast-Mapping of Volumetric Density Data onto a Two-Dimensional Screen 
25 The invention includes, in one aspect, a method for constructing from 

volumetric CT or MRl scan data of a patient region, a virtual two-dimensional 
fluoroscopic or shadowgraph image of the patient region, as seen from a selected 
point in space. The method includes the steps of selecting a point in space 
representing a virtual fluoroscopic irradiation source, and defining a plurality of 
30 rays extending between the source point and each of a plurality of points in a two- 
dimensional XY array of points, where (i) the anray points correspond to pixels in 
an XY pixel array in a display screen called a framebuffer, (ii) each pixel in the 
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display screen has multiple N-bit registers for receiving digitized values for each of 
multiple colors, (iii) the scan data consists of voxels having defined coordinates 
and associated M-bit tissue-density values, and (iv) at least some of the rays pass 
through a plurality of such voxels in the patient target region. 
5 For each ray, the bit positions of the M-bit voxel density values along that 

ray are distributed among the multiple N-bit registers of the associated pixel, such 
that one or more bit positions of each M-bit value is assigned to a selected register 
of that pixel. The M-bit density values of the voxels are summed along that ray by 
(i) individually summing the one or more bit-position values in each associated 

1 0 pixel register, and (ii) determining the sum of the M-bit values along the associated 
ray from the values in the individual registers. The image constnjcted of gray- 
scale values representing the summed M-bit density values at each pixel are 
displayed on the display screen. 

Where the pixel elements in the graphics screen each contain four 8-bit 

15 registers, corresponding normally to blue, green, red and alpha registers, the 
density values in each voxel are 8-bit numbers, and the bit positions of the 8 bit 
voxel-density values are distributed, for example, (a) two digits each to each of the 
registers, or 3 digits. 3 digits, 1 digit and 1 digit to the four registers. 

In a related aspect, the invention includes a system for constructing from 

20 volumetric CT or MRI scan data of a patient region, a virtual two-dimensional 
"fluoroscopic" or shadowgraph image of the patient region as seen from a 
selected point in space. The system includes a scan-data file for storing, in digital 
fomn, volumetric scan data of a surgical site of a patient, where the scan data 
consists of voxels having defined coordinates and associated M-bit tissue-density 

25 values, and a display screen containing a two-dimensional XY array of pixel 
elements, each element containing multiple N-bit registers for receiving digital- 
scale values for each of multiple colors. 

A computational device in the system is operatively connected to the data 
file and display screen for (i) defining a plurality of rays extending between a 

30 selected source point and each of a plurality of points in a two-dimensional XY 
array, where (ia) the array points correspond to the array pixels in the display 
screen, and (ib) at least some of the rays pass through a plurality of scan-date 
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voxels in the patient target region, (ii) for each ray, distributing the bit positions of 
the M-bit voxel density values among the multiple N-bit registers of the associated 
pixel, such that one or more bit positions of each M-bit value is assigned to a 
selected register of the associated pixel, (iii) for each ray, summing the M-bit 
5 values of the voxels along that ray by (iiia) individually summing the one or more 
bit-position values in each associated pixel register, and (iiib) determining the sum 
of the M-bit values along the associated ray from the values in the individual 
registers, and (iiic) plurality of passes over the previous step to further reduce a 
chance of register overflow and (iv) applying the summed tissue-density values 
1 0 along each ray to the associated pixel in the display screen. 

Virtual Fluoroscopic System and Method 

In another aspect, the invention includes a method of obtaining desired 
subsurface views of a patient target site. The method includes the steps of (a) 

1 5 obtaining volumetric scan data of a surgical site of a patient, and storing the scan 
data in digital form in a scan-data file, where the scan data is composed of voxels 
having defined coordinates and associated M-bit tissue-density values, (b) with the 
patient positioned in a surgical station, selecting a point in space representing a 
virtual fluoroscopic irradiation source, (c) defining a plurality of rays extending 

20 between the source point and each of a plurality of points in a two-dimensional XY 
array of points, where (i) the anray points correspond to pixels in an XY pixel array 
in a display screen, and (ii) at least some of the rays pass through a plurality of 
such voxels in the patient target region, (d) for each ray, summing the M-bit 
values of the voxels along that ray, (e) displaying the image constructed of gray- 

25 scale values representing the summed M-bit density values at each pixel in said 
display screen, and (f) repeating steps (a)-(e) until a desired displayed subsurface 
view of the patient target site is obtained. 

In a preferred embodiment, for rapid and real-time generation of images, 
the display screen contains a two-dimensional XY array of pixel elements, where 

30 each element contains multiple N-bit registers for receiving digital-scale values for 
each of multiple colors, and summing step (d) includes, for each ray, (i) distributing 
the M bits of the voxel density values among the multiple N-bit registers of the 
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associated pixel, sucfi tliat one or more bit positions of eacli M-bit value is 
assigned to a selected register of that pixel, and (ii) summing the M-bit values of 
the voxels along that ray by (iia) individually summing the one or more bit-position 
values in each associated pixel register, and (iib) determining the sum of the M-bit 
5 values along the associated ray from the values in the individual registers. 

Where the pixel elements in the graphics screen each contain four 8-bit 
registers, corresponding normally to blue, green, red and alpha registers, the 
density values in each voxel are 8-bit numbers, and the bit positions of the 8 bit 
voxel-density values are distributed, for example, (a) two digits each to each of the 

10 registers, or 3 digits, 3 digits. 1 digit and 1 digit to the four registers. 

The selecting step may be carried out using a hand-held pointer to Indicate 
the position and orientation of the desired view, where the position and orientation 
of the pointer can be tracked by a position-tracking device. The method, for 
example, allows for views along the long axis of the patient, i.e., views not 

1 5 othenA^ise available to the user. 

In a related aspect, the invention includes a system for use in obtaining 
desired subsurface views of a patient target site. The system includes (a) a scan- 
data file for storing, in digital form, volumetric scan data of a surgical site of a 
patient, where the scan data is composed of voxels having defined coordinates 

20 and associated M-bit tissue-density values, (b) a display screen containing a two- 
dimensional XY array of pixel elements, (c) a pointer for indicating selected 
positions and orientations from which subsurface views are to be reconstructed, 
and (d) a computational device. The latter is operatively connected to the data 
file, pointer, and display screen for (i) determining the position and orientation of 

25 the pointer, (ii) defining a plurality of rays extending between a selected pointer 
position and each of a plurality of points in a two-dimensional XY array, where (iia) 
the array points correspond to array pixels in the display screen, and (iib) at least 
some of the rays pass through a plurality of scan-date voxels in the patient target 
region, (iii) for each ray, for each ray, summing the M-bit values of the voxels 

30 along that ray, and (iv) displaying the image constructed of gray-scale values 
representing the summed M-bit density values at each pixel in said display screen. 
For use in real-time evaluation of the displayed images, the display screen 



5 



wo 02/19936 



PCTAJSOl/27715 



contams a two-dimensional XY an^y of pixel elements, where each element 
contains multiple N-bit registers for receiving digital-scale values for each of 
multiple colors. The computational device performs the summing operation for 
each ray by (i) distributing the M bits of the voxel density values among the 
5 multiple N-bit registers of the associated pixel, such that one or more bit positions 
of each M-bit value is assigned to a selected register of that pixel, and (ii) 
summing the M-bit values of the voxels along that ray by (iia) individually summing 
the one or more bit-position values in each associated pixel register, and (lib) 
determining the sum of the M-bit values along the associated ray from the values 
1 0 in the individual registers. 

Fluoroscopic Registration Svstem and Method 

In still another aspect, the invention includes a method of registering 
volumetric scan data of a patient surgical site with actual patient position. The 

15 method Includes the steps of (a) obtaining, at a scan station, volumetric scan data 
of a surgical site of a patient, including data relating to plural selected features in 
the region, and storing the scan data in digital form in a scan-data file, (b) after : 
moving the patient from the scan station to a surgical station, positioning an x-ray 
source and a detector screen for imaging at least a portion of said target region 

20 fluoroscopically at a selected orientation, (c) determining the coordinates of the x- 
ray source and detector at the positions in a fixed coordinate system, (d) activating 
the x-ray source to produce a fluoroscopic image of the patient region on the 
detector, (e) determining the image coordinates of the selected features in the 
target region in the fluoroscopic image in the fixed coordinate system, (f) using the 

25 coordinates determined in (c) and (e) to determine the actual coordinates of the 
selected features in the fixed coordinate system, and (g) matching the coordinates 
determined in (f) with those of the same features in the scan data, to place the 
scan data in the frame of reference of the patient. 

The method may further include using the scan data to construct a 

30 subsurface Image of the patient, as viewed from a selected position and 

orientation in the surgical station. Where the selected position and orientation is 
with respect to the tip of a surgical instmment, the method further includes tracking 
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the position of said instalment in the surgical station. 

In one embodiment, the plural selected features include at least three 
features, and steps (bHe) are carried out at different selected positions of the x- 
ray source and detector screen. 
5 In another aspect, the invention includes a system for registering volumetric 

scan data of a surgical site with actual patient position. The system includes (a) a 
scan-date file for storing, in digital fomri, volumetric scan data of a surgical site of a 
patient, including data relating to plural selected features in the region, (b) an x-ray 
source and a detector screen for imaging at least a portion of the target region 

1 0 fluoroscopically at a selected orientation, (c) a fixed-position sensor for 

detemnining the coordinates of the x-ray source and detector at the positions in a 
fixed coordinate system, and (d) a computational device operatiyely connected to 
the data file, detector screen, and fixed-position sensor, for (i) detemnining the 
actual coordinates of selected features of the surgical site in the fixed coordinate 

1 5 system from the determined coordinates of such features on the detector screen 
and from the coordinates of the screen, and (ii) matching the actual coordinates of 
the patient features with those of the same features in the scan data, to place the 
scan data in the frame of reference of the patient. 

These and other objects and features of the invention will become more 

20 fully apparent when the following detailed of the invention is read in conjunction 
with the accompanying drawings. 

Brief Description of the Drawings 

Fig. 1 illustrates an arrangement of components of a surgical imaging 
25 system and a patient in a surgical setting containing the system; 

Fig. 2 is a flow diagram of steps executed by a computational device in the 
system, for registering pre-op scan data with actual patient position, and for using 
the scan data to construct target images from a selected position and orientation; 

Fig. 3 shows the projection of rays through a target region onto a virtual 
30 array from a virtual source point, in the method of the invention for constmcting 
virtual fluoroscopic images; 

Fig. 4 illustrates the use of voxel elements in calculating pixel density 

7 
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values, in constructing a virtual fluoroscopic innage in accordance with the 
invention; 

Fig. 5 shows four registers associated with each pixel element in a display 
screen employed in the present invention; 
5 Fig. 6 shows one assignment of density-value digits to display-screen 

registers in the method of the invention; 

Fig. 7 is a flow diagram of the fast mapping method in accordance with the 
invention; 

Fig. 8 illustrates the virtual fluoroscopy method of the invention, showing a 
1 0 virtual fluoroscopic source at two selected points; 

Figs. 9A and 9B show views of a patient spinal column as generated by 
virtual fluoroscopy from the two virtual points shown in Fig. 8; and 

Fig. 10 is a flow diagram of the steps in the virtual fluoroscopy method of 
the invention. 

15 

Detailed Description 
The present invention is designed to enhance several aspects of image- 
guided surgery. In each case, the invention depends on or uses pre-operative 
volumetric scan data of a patient target region. Typically, the scan data acquired 

20 are computerized tomography (CT), magnetic resonance imaging (MRI), or 

ultrasound scan data, obtained according to well-known methods. The scan data 
are conventionally stored in digitized form in a scan-data file, for later use in the 
invention. For purposes of the present invention, the scan data have three 
important characteristics. 

25 First, the target region (volume) that is scanned is divided into a three- 

dimensional voxel space, where each voxel (small volume element) typically 
represents an x-y element in the "plane" of scans taken along the z direction. 
(Voxels defined by other types of coordinate systems, e.g., polar coordinates, 
may also be employed). Second, each voxel is associated with an M-bit density 

30 value that characterizes the scan density observed for that volume element in 
space. Third, the voxel coordinates are placed in a patient coordinate system 
keyed to one or more of the following: (I) patient structural features, such as 

8 
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bone features or external surface features, (ii) fiducials, typically small metal 
markers, implanted at selected positions within the patient target region, (iii) 
fiducials placed at fixed positions on the surface of a patient, or (iv) sensor 
elements placed at fixed positions in or on the patient. 
5 During a surgical procedure-that is, after the patient has been moved 

from a scanning station to a surgical station-the scan data must be placed in 
the new patient coordinate system. This is done, in general, by determining the 
coordinates of patient physical structure, e.g., bone features or fiducials, and 
transforming the coordinates of the scan-data voxels to the new coordinate 
10 system. 

Various approaches to placing pre-op scan data in registration with the 
patient surgical position have been proposed heretofore. In one, fiducials whose 
position in space can be determined by a fixed-position sensing unit are placed 
on the patient. The fiducial positions in the surgical setting can then be 

15 detemnined directly by the sensing unit, and used as reference points for the 
scan data, which Includes images of the fiducials at known voxel positions. The 
limitation with this method, at least in spinal surgery, is that the fiducials must be 
placed on the patient surface, and typically do not provide accurate coordinates 
for subsurface structure, e.g., vertebrae. 

20 In a second approach, a fluoroscopic Image of the patient target region is 

produced on a display screen, and this image is then position-matched with a 
digitally reconstructed radiograph (DRR) produced by reconstructing scan data. 
By manipulating the coordinates of the scan data until the DRR and actual 
fluoroscopic image match, scan data coordinates corresponding to actual patient 

25 coordinates are obtained. This method is described, for example, in U.S. Patent 
Nos. 4,791 ,934 and 5,901 ,199. One aspect of the present invention, described 
in Section II below, is a fast-mapping algorithm that significantly reduces the 
time needed to generate virtual radiographic images, and thus significantly 
improves this image-matching method. 

30 A third approach, which forms another aspect of the present invention, is 

described in Section I below. 
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I. Registering Pre-op Scanning Data with Patient Position 

Figure 1 illustrates components of a system 10 of the invention, shown in 
operative relationship to a patient 12 placed face-down on a surgical table 14. 
For purposes of illustrating the describing the invention, the patient region of 
5 interest-the surgical target region-is the patient's spine, indicated at 16. 
Although the invention is applicable to any patient target region, the invention is 
particularly applicable to spinal region, both because the vertebrae of the spine 
offer a good fluoroscopic target (bone), and because the position of the spine is 
difficult to track by external markers when the patient moves or is moved. The 
10 same applies to any piece-wise rigid structure (e.g., any part of skeletal 
anatomy). 

System 10 includes a data-scan file 18 which stores pre-op scan data, 
typically obtained at a scanning station (not shown) which is different from the 
surgical station shown. As indicated above, the scan-data consists of voxel 

1 5 coordinates, corresponding scan-data, e.g., density values, for each voxel, and 
coordinates for registration features, e.g., selected bone features or fiducials. 
The scan data file is operatively connected to a computational device 20, e.g., 
computer or processor, which carries out the data processing operations of the 
system, to be described. Also connected to the computer are a fixed-position 

20 sensing unit 22 and a display device 24. The display device is preferably 
includes a conventional computer color-monitor screen composed of a 
framebuffer with an XY array of pixels, where each pixel has four N-bit registers, 
e.g., 8-blt registers, con^esponding to red, green, blue and alpha registers. 

Sensing unit 22 is part of a position tracking system, which is preferably 

25 an optical tracking system (hereafter "OTS") having sensing unit 22 mounted 
overhead in view of the operating table scene, and various movable objects, 
each equipped with at least two light emitting diodes (LED's) mounted on the 
object. In the embodiment shown, the movable objects include first and second 
detecting plates 30, 32 designed to be placed at selected positions adjacent the 

30 patient, as shown, and an x-ray source 38 also designed to be placed at 

selected position with respect to the patient. Another movable object that may 
be included in the system, but which is not necessarily used for registration, is a 

10 
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surgical instrument 26 designed to be used by the physician during a surgical 
procedure. 

As shown, X-ray source 38 is equipped with LED's 40; screen 30, with 
LEDs 34; screen 32, with LEDs 36, and instrument 26, with LEDs 28. The 
5 LED's attached to the movable objects are preferably designed to emit streams 
of pulsed infrared signals which are sensed by a plurality of infrared detectors 
(not shown) contained in the sensing unit. The sensing unit and movable 
objects are connected to the computer, which controls the timing and 
synchronization of the pulse emissions by the LED's and the recording and 

10 processing of the infrared signals received by the detectors. The OTS further 
includes software for processing these signals to generate data indicating the 
location and orientation of the movable objects in the frame of reference of 
surgical station. The OTS may generate the position data on a real-time basis, 
so that as the surgical instalment is moved, for example, its position and 

15 orientation are continually tracked and recorded by the sensing unit, and when 
the display screens or x-ray source are moved, their positions in the coordinate 
system of the surgical station are known. 

The OTS is preferably of the type known as the "FlashPoint 3-D Optical 
Localizer", which is commercially available from Image Guided Technologies of 

20 Boulder, Colorado, similar to the systems described in U.S. Patent Nos. 

5,617,857 and 5,622,170. However the invention is not limited to this particular 
OTS, and other position tracking systems, such as sonic position detecting 
systems, may also be utilized. 

X-ray source 38 may be, for example, any of a number of commercially 

25 available fluoroscopic sources such as an OES 9600 obtained from General 
Electric Medical Systems (Milwaukee, Wl). The source is typically mounted on a 
suitable frame (not shown) for positioning at a selected "view" position with 
respect to the patient. The detecting plates may be, for example, conventional 
rectangular arrays of CCD devices or other electronic components that convert 

30 radiation intensity into corresponding electronic signals. The CCD elements in 
the plates are operatively connected to conresponding pixel elements in the 
display screen, for displaying the fluoroscopic or shadowgraph image of the 



11 



wo 02/19936 



PCT/USOl/27715 



patient region projected onto the detecting plate. For example, if the CCD 
devices output an 8-bit digital signal, this output can be supplied to one of the 
four display screen registers to display the CCD array output on the display 
screen. Also as indicated in the figure, the CCD elements making up the arrays 
5 in each detecting screen are connected to computer 20, for supplying the 
computer with digitized fluoroscopic data recorded at the array positions on the 
plate. 

Ultimately, and as described further below, the system will be used to 
display patient target-region images reconstructed from the scan data and as 

10 seen, for example, from the position and orientation of instrument 26. for guiding 
the physician during a surgical operation. 

The registration method of the invention is illustrated in Fig. 1. and the 
steps in practicing the method, including those steps executed by computer 20, 
are given in the flow diagram in Fig. 2. Initially, source 38 and plate 30 are 

1 5 positioned at selected positions to image the patient target region, e.g., spinal 
region, from a first selected position, as illustrated in Fig. 1 . With the source and 
plate so positioned (the source position is indicated at Pi) the OTS is activated to 
record the positions of both source and plate in the frame of reference of the 
sensing unit, that is, with respect to a fixed point of reference within the surgical 

20 station. The X-ray source is then activated to produce a fluoroscopic image or 
shadowgraph of the patient region, as detected at the detector plate. The array 
signal intensity data is supplied to display 24, for displaying the fluoroscopic image 
to the user, and also input into a dedicated image file in computer 20. 

At this point, the system (with or without the user's assistance) identifies 

25 three or more patient-region features that are also recognizable in the pre-op scan 
data. The features may be, for example, radio-opaque fiducials, which can be 
identified by their high-density values, or bone features that can be identified by 
their distinctive shape or position. The user may specify these features by 
"clicking" on these features in the image displayed in device 24, or the features 

30 may be "recognized" by the system automatically according to distinctive density 
values, e.g., by sharp density-edge effects. Once the features have been 
identified, the system records the XY positions of the features on the display 
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screen, corresponding to the XY positions on the detector plate. 

The source and detector plate are now shifted to second selected positions 
for viewing the target region from a second angle. In the embodiment shown, 
source 38 is moved from Pi to P2, and the detector plate (which may be the same 
5 plate 30 or a second plate 32) is moved to the position shown for plate 32. A 
second fluoroscopic image of the target region is then taken, with the image being 
displayed to the user at device 24 and stored in the processor as above. Again 
the system or user identifies the same target features identified earlier, and the 
positions of these features on the detector plate are stored. 

10 The program flow in Fig. 2 shows how the system determines the actual 

spatial coordinates of the patient features, e.g., features a, b in Fig. 1. Initially the 
spatial coordinates of the detector plate(s) at both views are determined by the 
OTS and stored in the computer, as at 42. Source 38 is then positioned at 
selected position Pi, as at 44, and its coordinates determined by the OTS. as at 

15 46. The source is activated to produce a fluoroscopic image or shadowgraph on 
detector plate 30. The positions of features a, b in the shadowgraph are indicated 
at Fia and Fib in Fig. 1 , corresponding to positions xia,yia and xib,yib on the 
detector plate an'ay, and to pixels x'la./ta and x'ib,y'ib on the display screen. From 
the known x,y positions of features a, b on plate 30, and from the known position 

20 of the plate, the computer detenmines the coordinates of Fia and Fib (xia,yia and 
xib.yib) by standard geometric calculations, as at 48, and these coordinates are 
stored in the computer for later use. 

Similarly, after moving the source to selected position P2, as at 50, and 
determining the new source position, as at 52, and with detector plate 32 at a 

25 second selected plate position, the user takes a second fluoroscopic image of the 
patient region from this second selected view. The second image contains the 
features of interest, e.g., a, b, as indicated at F2a and F2b in Fig. 1 . As above, from 
the known positions of these features on the detector plate, and from the OTS 
determined position of the plate, the coordinates of F2a and F2bare determined, as 

30 at 54. 

The program now has the x,y,z spatial coordinates for the two source 
positions Pi and P2, and for the plate coordinates of the patient features on both 
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plates (or one plate at two selected position), e.g., Fia and Fib and F2a and Fab. 
These coordinates allow the program to calculate, by standard "intersection" 
equations, the x,y,z coordinates of selected patient features, e.g., a, b, as at 56. 
The system next determines the scan-data coordinates for the same 
5 patient features in the scan data file 18. This may be done, in one embodiment, 
by reconstructing a virtual image from the scan data, and having the user "click 
on" the feature(s), e.g., implanted fiducials or distinctive bone features, used as 
landmarks in the fluoroscopic images, e.g., features a, b. Alternatively, the 
program may automatically identify the landmark features by distinctive density 

10 values or density patterns, as above. 

Once the selected features or landmarks in the scan data are identified, the 
program will construct a transfonn that transforms the internal coordinates of 
these features into the spatial coordinates of the features detemiined from the 
fluoroscopic images, as at 58. This transfomi, when applied to all of the scan-data 

1 5 coordinates, effectively places the scannjata in the actual frame of reference of 
the patient. That is, the scan data is placed in registration with actual patient 
position. 

After executing this transformation, the system can use the scan data to 
construct selected subsurface views, e.g., perspective views) of the patient target 

20 region, corresponding to the actual patient position. These images may be 

presented to the physician at display device 24. In one preferred embodiment, the 
OTS in the system is used to track the real-time position of a surgical tool or 
instrument, e.g., instrument 26, and the reconstructed images are be displayed 
from the position and orientation of the instalment, allowing the user to view 

25 hidden tissue and the instrument is moved toward or within the patient target site. 
Methods and algorithms for producing such images, particularly as seen from the 
point of view of a movable instrument, are described, for example, in PCT 
application WO 9900052 A1 for Image generation of three dimensional object, 
which is incorporated herein by reference. 
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11. Fast 3-D to 2-D mapping method and system 

In another aspect, the invention provides a rapid method for mapping 3-D 
scan data onto a 2-D display format, particularly for use in generating digitally 
reconstructed radiographs (DDRs) or virtual shadowgraphs from CT, MR! or 
5 ultrasound volumetric scan data. Such radiographs are also referred to herein as 
virtual fluoroscopic images or virtual shadowgraphs, meaning a displayed image 
that resembles a fluoroscopic image taken from a selected view, but which is 
constructed from pre-op scan data rather than actual x-irradiation of the patient 
site. • 

1 0 The system for canrying out the invention generally Includes the 

components shown In Fig. 1 , in particular, a scan data file 18, computational 
device 20, and display device 24. As described above, the scan file consists of 
voxel coordinates, corresponding scan-data, e.g., density values, for each voxel, 
and coordinates for registration features, e.g., selected bone features or 

15 fiducials. The scan data file is operatively connected to the computational 
device, e.g., computer or processor, which carries out the data processing 
operations of the system to be described. 

The display device is preferably a conventional computer color monitor 
having a display screen composed of an XY array of pixels, e.g., a 1280 X 1024 

20 pixel array. The framebuffer provides, for each pixel, a plurality of N-bit 
registers, typically four 8-bit registers corresponding to blue, red, green, and 
alpha single-valued pixel input values. Each pixel register functions to accept an 
N-bit color value, for outputting to the corresponding pixel, an electronic signal 
indicating the corresponding brightness of that color in that pixel. According to 

25 an important requirement of the invention, each register is hardwired with an 
"add" function that allows the register to add successive values (with N or fewer 
bits) supplied to the register. Virtually all commercially available color monitors 
have these features and are therefore suitable for use in the invention. 
In the present system, the computer is connected to the monitor to 

30 provide input to all of the pixels, and to each of the registers in each pixel 

element, simultaneously, to allow parallel input to each pixel during the operation 
of the system. The operation of the computer, and the computer software or 
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hardwired program required in the system in the system will be appreciated 
below from the description of the method carried out by the system, with 
reference to Figs. 3-7. 

It is assumed in the following discussion that it is desired to construct a 
5 virtual fluoroscopic image of a spinal target region 16 in patient 12, and that the 
view of the target tissue is that which would be produced by projecting x-rays 
from a selected source point Pi onto a detector screen placed at the position 
occupied in the figure by a virtual detector plate, indicated at 64. The virtual 
plate is considered as an array of points, such as points 64, 66, corresponding 

10 to the CCD elements in an actual detector plate array, and each point is 
designated by a unique x,y coordinate in the an^ay. That is, all points in the 
array can be represented by coordinates xi.yk, i=1 to m, and j=1 in an m X n 
array. Further, the points correspond, through a one-to-one mapping, to the 
pixels in the XY pixel anray of the display device. 

15 In practicing the method, the user selects a desired source point Pi from 

which the patient target tissue will be viewed, as at 76 in Fig. 7. This may be 
done, for example, by providing a movable frame-mounted pointer that can be 
moved to a selected point with respect to the patient, allowing the field of view 
angle to be selected according to the distance between and pointer and the 

20 patient, and the view orientation to be selected according to the orientation of 
the pointer. The position and orientation of the pointer may be tracked by the 
above GTS (or comparable tracking device) to indicate to the computer the 
actual coordinates of the virtual source point Pi. In particular, the pointer can be 
rigidly fixed to the exterior of the real fluoroscopic camera to allow for natural 

25 simulation of its use. Similarly, it can be hand-held to permit computing views 
that would not be possible to physically realize with a real fluoroscope (e.g., axial 
views). 

With virtual source point P1 selected, the computer constructs a set of 
rays as at 78 in Fig. 7. Figs. 3 and 4 show exemplary rays ^^ and r2 in such a 
30 construction. The computer selects the coordinates of the virtual detector points 
so that at least a portion of the rays pass through the target region of interest, as 
shown in Fig. 3. Preferably the cross-section of the rays encompasses the 
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entire target region, and a central ray is nonmal to the plane of virtual plate. 

In constructing the virtual fluoroscopic image, the program operates to 
sum the scan-data density values along each ray in the set of rays, and place 
this sum (after scaling) in one of the registers, e.g., the gray-scale register of the 
5 corresponding pixel in the display device, to display the virtual fluoroscopic 
image to the user. In order to perform this summing operation, the program 
must define a frame of reference for the scan data, so that each voxel in the 
scan data can be assigned to one or more rays projecting through the patient 
target region. If the scan data has been placed in registry with the patient 

10 position, as described above, the coordinates of the scan data voxels can be 
assigned actual patient coordinates in the surgical setting. If no such 
registration is performed, or if the patient moves after the registration operation; 
the voxel coordinates may con^espond approximately, but not precisely, to actual 
patient coordinates. 

1 5 Fig. 4 illustrates the assignment of voxels to rays r1 and r2. as 

: representative of all of the rays considered in the method. The computer knows 
• the actuarx,y,z coordinates of P1 , and has assigned x,y coordinates to each of 
the points in the virtual detector array. (The z axis is considered here as the 
axis normal to the plane of the virtual detector array). Therefore, the x,y position 

20 of each ray at each scan slice (along the z axis) is known and assigned a voxel 
corresponding to the x,y,z coordinates of the ray at that slice position (z 
coordinate). As noted above, the voxel coordinates may be actual patient 
coordinates, or may be coordinates assigned when the scan data was obtained. 
For some rays, such as ray ri, the ray may extend through a single column of 

25 voxels in the patient target region, such as the voxels identified in Fig. 4 as V1p, 
where p=1 to w; for others, such as ray r2, the ray may extend through two or 
more voxel columns, such as the columns identified by voxels \/2p (p=1 to y) and 
the adjacent voxel column made up of voxels Vaq, q=1 to x). Alternatively, the 
voxels may be assigned polar coordinates, such that each ray passes through a 

30 single column of voxels. 

Each of the voxels is associated with a tissue-density value representing 
the tissue density seen by the scanning device at that voxel position. The 
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density value is scaled, if necessary, to a selected M-bit number, preferably an 
8-bit number to represent, for example, 255 different density values. The 
program then operates to sum the density values along each ray, to produce a 
summed value representing the total tissue density that an x-ray would 
5 encounter on traversing the associated column(s) of voxels in the patient target 
region. 

The summing operation along the rays is carried out in parallel for all 
rays, whereby the total summing operation time is no greater than that required 
for a single ray. According to an important feature of the invention, the summing 

1 0 operation is further accelerated by taking advantage of the ability of the pixel 
registers to sum values applied to the registers. IHow this feature is exploited in 
the present invention is illustrated in Figs 5 and 6. Fig. 5 shows a portion of a 
display screen 70 having an XY array of pixels, such as pixels 72, 74 
corresponding to the XY array of points in the virtual detector array in Figs. 3 
. 1 5 and 4. Each pixel, such as pixel 72, receives input from four N-bit registers, 
such as registers 72a, 72b, 72c, and 72d associated with pixel 72 and registers 
74a, 74b, 74c, and 74d associated with pixel 74. In the normal color-display 
mode of the display device, the four registers operate to signal impart a 
particular color to each pixel, depending on the relative values supplied to the 

20 pixel by the four registers. 

Typically, the pixel registers are 8-bit registers, and therefore capable of 
holding up to 255 different digitized values. In the discussion below, 8-bit 
registers will be assumed, although it will be recognized that the summing 
operation could be applied to larger registers, e.g., 16-bit registers. 

25 For each ray, the summing operation is carried out by first considering the 

8-digit density value as four separate 1-, 2-, or 3-bit numbers. The reason for 
splitting the input values this way is to reduce the possibility that any of the 
individual registers will overflow. The four smaller values, representing particular 
bit positions in the 8-bit number, are then assigned to the four color registers. 

30 For example, in the assignments illustrated in Fig. 6, the 8-bit density-value 
number is broken down to four two-bit numbers, such that the 8-bit number 
n7n6n5n4n3n2nino becomes njne ,n5n4 , n ana , and nino, with each of the 2-blt 
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numbers being assigned to one of the four registers. Another preferred 
assignment considers the 8-bit number in two groups or three digits and two 
groups of one digit, such that the 8-bit number n7n6n5n4n3n2nino become nynens, 
n4n3n2, ni, and no, again with the four smaller numbers being assigned to the 
5 four pixel registers. An advantage of the later bit distribution is that lowest-bit 
numbers, e.g., ni, and no, are expected to include higher noise values, and thus 
may overflow the associated registers faster during a summing operation, than 
do the higher-bit numbers. By limiting the lower-bit numbers to one bit positions, 
the tendency to overflow the associated registers is minimized. 

10 In a summing operation, illustrated in Fig. 6, the first voxel 8-bit number In 

each ray (n=1 ) is read, as at 80 in Fig. 7, and is placed in the four registers of 
the con'esponding display pixel, in accordance with the above bit assignments, 
such that each register receives a 1-, 2-, or 3-bit number, as at 82 and 84 in Fig. 
7. The program then proceeds to the next voxel in each ray (n=n+1), as at 86. 

1 5 and repeats the above steps by adding the corresponding 1-, 2-, or 3- bit 
number to the same registers. As noted above, the registers are hardwired to 
perfonm number addition, so the addition operation occurring at all of the 
registers can take place fill rates of 1 Gpixels/second. At each addition step, 
and for each ray in parallel, the program adds the next 8-bit voxel value to the 

20 associated four registers, and this process is continued until all voxels along all 
rays have been considered, as at 88. 

To obtain the final density value for each pixel, the values of the four 
registers are summed, preserving the order of the original assignment of digits, 
as at 90 in Fig. 7. That is, any carry over from the register containing the lowest- 

25 value digits is added to the register containing the next-higher-value bits, which 
is added to the register with the next-higher-value bits and so forth. The final 
summed value, which will typically be a 24-32 bit number, may then be scaled to 
an 8-bit number, as at 92 in Fig. 7, and placed in the screen's gray-scale 
registers, to display the virtual image in gray scale, as at 94 in Fig. 7. 

30 From the foregoing, various advantages and features of the invention can 

be appreciated. The mapping method, because it utilizes parallel addition in a 
slice-by-slice manner along the scanned tissue volume, and takes advantage of 
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the hardwired addition feature of the display screen registers, is approximately 
two orders of magnitude faster than would be achieved by treating the four color 
registers in each pixel as a 32-bit register, and successively summing 8-bit 
numbers in this expanded register. This is due to inherent parallelism of the 
5 method (all rays are operated on simultaneously) and straightfonA^ard 

implementation on off-the-shelf hardware. More realistic light transport models 
(e.g., involving X-ray scattering) can also be incorporated in this model. 

The speed of operation allows for virtual fluoroscopic image generation in 
real time, in accordance with the method described in Section III below. For 

10 example, accumulating 512 values along each ray on a 512^ viewport requires 
512^ or 128 MegaAdditions, hence a graphics adapter with 1 Gpixel fill rate can 
compute 8 DRRs in a single second. It also facilitates other digitally 
reconstructed radiograph (DDR) applications, such as the one described in the 
background of Section I. In one embodiment, the invention includes an 

15 improved registration method, such as described in U.S. Patent No. 4,791 ,934, 
where the DRR is generated by the fast-mapping method described herein. 
Another application of the fast-mapping method is the virtual fluoroscopic 
method now to be discussed. 

20 IIL Virtual Fluoroscopic Imaging 

In still another aspect, the invention provides a virtual fluoroscopic method 
and system for "imaging" subsurface patient tissue, e.g., the spinal region, during 
a surgical or diagnostic procedure on the patient. The aim of the method, in part, 
is to allow a surgeon to view fluoroscopic-like or shadowgraph-like images of a 

25 patient target region on a frequent or repeated basis, without exposing the patient 
and others in the surgical station to x-radiation. For example, when doing spinal 
surgery, the surgeon may need to make frequent checks on the point of entry of a 
tool, or the placement of an implant, or the angle or depth of a cut, by referring to 
a fluoroscopic image of the region. The present method allows frequent virtual 

30 fluoroscopic images to be generated from a selected point in the surgical station. 
In a preferred embodiment of the method, employing the fast-mapping 
algorithm described in Section II, the surgeon can generate real-time images as 
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the virtual view position and orientation is adjusted. This allows the surgeon, for 
example, to move the virtual view point about the patient until an optimal 
fluoroscopic view of the surgical site is obtained, and then take an actual 
fluoroscopic image from the optimal position and orientation. 
5 The system for carrying out the invention generally includes the 

components shown in Fig. 1 and in particular, a scan data file 18, computational 
device 20, and display device 24. As described above, the scan file consists of 
voxel coordinates, corresponding scan-data, e.g., density values, for each voxel, 
and coordinates for registration features, e.g., selected bone features or 

10 fiducials. The scan data file is operatively connected to the computational 
device, e.g., computer or processor, which carries out the data processing 
operations of the system to be described. 

Optionally, the system may contain a tracking system, such as the OTS 
■ described in Section I, which may include either or both of two user-held 

1 5 instruments discussed below with reference to Figs. 8 and 9. 

As in the system described in Section II, the display device is preferably a 
conventional computer color monitor having a display screen composed of an 
XY array of pixels, e.g., a 1280 X 1024 pixel array. The monitor provides, for 
each pixel, a plurality of N-bit registers, typically four 8-bit registers 

20 corresponding to blue, red, green, and alpha (gray-scale) pixel input values. 

Fig. 8 illustrates a surgical setting in which a patient 12 placed on a surgical 
bed 14 is undergoing spinal surgery at the target region indicated at 16. The 
figure shows a hand-held pointer 100 having tracking members, such as LEDs 
101, for tracking the position and orientation of the pointer in space, through a 

25 suitable sensing device (not shown) such as the OTS described above. In the 
setting shown, it is assumed that the surgeon wishes to find an optimal view 
position and orientation for a fluoroscopic image, or simply wishes to view target 
structure by fluoroscopic-like images, e.g., for purposes of surgical planning. 

In the method, the surgeon initially selects a desired point Pi, indicated at 

30 102, and view orientation, as at 1 12 in Fig. 10. This is done, for example, by 
moving pointer 100 to a selected position and orientation, or by moving a frame- 
mounted pointer to a selected position and orientation. Upon user-activation, the 
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system will then generate a virtual fluoroscopic image of the target tissue from the 
selected point and view orientation. 

The system operates, as at 1 14 in Fig. 10, to generate the virtual image 
substantially by the method described in Section II, with reference to Figs. 3 and 4, 
5 although not necessarily employing the fast-mapping algorithm described in 
Section II. Briefly, the system constructs a set of rays from the selected source 
point to the points in a virtual detector array, such as the virtual detector indicated 
at 104, and detemnines the sum of voxel density values along each ray from the 
scan data. The sum along each ray is then scaled, e.g., to an 8-bit number, and 

10 these values applied to the conresponding pixels in the display device, to generate 
a reconstructed shadowgraph image. As in Section II, the voxel coordinates may 
be precisely registered with the patient frame of reference, as at 1 18 in Fig. 10, in. 
which case the reconstructed image precisely matches an actual fluoroscopic 
image from that position and orientation. If the scan data is not precisely 

1 5 registered with patient coordinates, the reconstructed image will be approximate 
only as to actual position. 

In a preferred embodiment, the summing method applied to the voxel 
density values employs the fast mapping method described in Section II, where 
the summing operations are carried out by the individual registers up to the final 

20 summing of the values in the four different pixel registers. This method, because 
of its speed, allows reconstmcted shadowgraph images in real time, as the user 
moves the pointer around in space. This allows the surgeon to quickly and 
efficiently find optimal views from which actual fluoroscopic images can be taken, 
or, if the scan data is in registry with patient coordinates, to view patient 

25 substructure accurately from continuously varying points of view, that is, with true 
x-ray vision. 

Fig. 8 shows selected views at points PI (point 102) and P2 (point 106), 
and the virtual detector plates 104 and 108 associated with each view, as at 1 16 
in Fig. 10. Figs. 9A and 9B show corresponding reconstructed shadowgraph 
30 images of a patient spinal region seen from points Pi and P2, respectively, in 
accordance with the invention. Also shown in the figures is a surgical instrument 
1 10 employed during a surgical position. Sensor elements, such as LEDs 1 1 1 on 
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the instrument, are used for instrument tracking, as at 120 in Fig. 10, allowing tfie 
computer to reconstruct a virtual image of the instrument from its known position 
and superimpose this image on the reconstructed shadowgraph. In particular, 
with the scan data in registry with the patient coordinates, the user can view both 
5 hidden patient structure and the features of an instrument being employed in a 
surgical procedure, as at 122 in Fig, 10. Altematively, or in addition, the surgical 
instrument can act as a pointer, to allow the surgeon to view target tissue from the 
position and orientation of the tool, to augment the information provided by the 
virtual fluoroscopic method. 

10 Although the invention has been described with respect to particular 

components, operations, and applications, it will be appreciated that various 
■ modifications and changes may be made without departing from the invention as 
claimed. For example, a variety of tracking system may be employed from 
tracking the positions and orientations of various pointers, instruments and 

1 5 recorder plates In the surgical station, without affecting the overall result or 
operation of the claimed methods and systems. 



23 



wo 02/19936 PCT/USOl/27715 

What is claimed is: 

1 . A method for constructing from volumetric CT or MR! scan data of a 
patient region, a virtual two-dimensional fluoroscopic or shadowgraph image of the 

5 patient region as seen from a selected point in space, comprising 

(a) selecting a point in space representing a virtual fluoroscopic irradiation 
source, 

(b) defining a plurality of rays extending between the source point and each 
of a plurality of points in a two-dimensional XY array of points, where (i) the array 

1 0 points correspond to pixels in an XY pixel array in a display screen, (ii) each pixel 
in the display screen has multiple N-bit registers for receiving digitized values for 
each of multiple colors, (iii) the scan data is composed of voxels having defined 
coordinates and associated M-bit tissue-density values, and (iv) at least some of 
the rays pass through a plurality of such voxels in the patient target region, 

15* (c) for each ray, distributing the M bits of the density values of voxels along 
that ray among the multiple N-bit registers of the associated pixel, such that one or 
more bit positions of each M-bit value is assigned to a selected register of that 
pixel, 

(d) for each ray, summing the M-bit values of the voxels along that ray by (i) 
20 individually summing the one or more bit-position values in each associated pixel 

register, and (ii) detemninlng the sum of the M-bit values along the associated ray 
from the values in the individual registers, and 

(e) displaying the image constructed of gray-scale values representing the 
summed M-bit density values at each pixel in said display screen. 

25 

2. The method of claim 1 , wherein the pixels in the display screen each 
contain four 8-bit registers, the density values in each voxel are 8-bit numbers, 
and the 8 bit voxel-density numbers are distributed two digits each to each of the 
registers. 

30 

3. The method of claim 1 , wherein the pixels in the display screen each 
contain four 8-bit registers, the density values in each voxel are 8-bit numbers, 
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and the 8 bit voxel-density numbers are distributed 3 digits, 3 digits, 1 digit and 1 
digit to each of the four registers, 

4. A system for constructing from volumetric CT or MR! scan data of a 

5 patient region, a virtual two-dimensional fluoroscopic image of the patient region 
as seen from a selected point in space, comprising 

(a) a scan-data fife for storing, in digital fomri, volumetric scan data of a 
surgical site of a patient, where the scan data is composed of voxels having 
defined coordinates and associated M-bit tissue-density values, 

10 (b) a display screen containing a two-dimensional XY array of pixel 

elements, each element containing multiple N-bit registers for receiving digital- 
scale values for each of multiple colors, and 

(c) a computational device operatively connected to the data file and 
display screen for (i) defining a plurality of rays extending between a selected 

1 5 source point and each of a plurality of points in a two-dimensional XY an^ay, where 
■' (ia) the an"ay points correspond to an-ay pixels In the display screen, and (ib) at 
least some of the rays pass through a plurality of scan-date voxels in the patient 
target region, (ii) for each ray, distributing the M bits of the voxel density values 
among the multiple N-bit registers of the associated pixel, such that one or more 

20 bit positions of each M-bit value is assigned to a selected register of that pixel, (iii) 
for each ray, summing the M-bit values of the voxels along that ray by (iiia) 
individually summing the one or more bit-position values in each associated pixel 
register, and (iiib) determining the sum of the M-bit values along the associated 
ray from the values in the individual registers, and (iv) applying the summer tissue- 

25 density values along each ray to the associated pixel in the display screen. 

5. A method of obtaining desired subsurface views of a patient target site, 
comprising 

(a) obtaining volumetric scan data of a surgical site of a patient, and storing 
30 the scan data in digital form in a scan-data file, where the scan data is composed 

of voxels having defined coordinates and associated M-bit tissue-density values, 

(b) with the patient positioned in a surgical station, selecting a point in 
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space representing a virtual fluoroscopic irradiation source, 

(c) defining a plurality of rays extending between the source point and each 
of a plurality of points in a two<limensional XY anray of points, where (i) the array 
points correspond to pixels in an XY pixel array in a display screen, and (ii) at least 

5 some of the rays pass through a plurality of such voxels in the patient target 
region, 

(d) for each ray, summing the M-bit values of the voxels along that ray, 

(e) displaying the image constaicted of gray-scale values representing the 
summed M-bit density values at each pixel in said display screen, and 

10 (f) repeating steps (aHe) until a desired displayed subsurface view of the 

patient.target site is obtained. 

^ 6. The method of claim 5, for use in real-time evaluation of the displayed 
images, wherein the display screen contains a two-dimensional XY aray of pixel 

1 5 elements, each element containing multiple N-bit registers for receiving digital- 
scale values for each of multiple colors, and summing step (d) includes, for each 
ray, (i) distributing the M bits of the voxel density values among the multiple N-bit 
registers of the associated pixel, such that one or more bit positions of each M-bit 
value is assigned to a selected register of that pixel, and (ii) summing the M-bit 

20 values of the voxels along that ray by (iia) individually summing the one or more 
bit-position values in each associated pixel register, and (iib) determining the sum 
of the M-bit values along the associated ray from the values in the individual 
registers: 

25 7. The method of claim 6, wherein the pixels in the display screen each 

contain four 8-bit registers, the density values in each voxel are 8-bit numbers, 
and the 8 bit voxel-density numbers are distributed two digits each to each of the 
registers. 

30 8. The method of claim 5, which further Includes the step, prior to step (a) 

of placing the scan data coordinates is registry with actual patient coordinates. 
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9. The method of claim 5, wherein said selecting step (b) includes using a 
hand-held pointer to indicate the position and orientation of the desired view, 
where the position and orientation of the pointer can be tracked by a position- 
tracking device. 

5 

10. The method of claim 5, wherein said selecting includes selecting views 
along the long axis of the patient. 

1 1 . A system for use in obtaining desired subsurface views of a patient 
1 0 target site, comprising 

(a) a scan-data file for storing, in digital form, volumetric scan data of a 
surgical site of a patient, where the scan data is composed of voxels having 

: defined coordinates and associated M-bit tissue-density values. 

(b) a display screen containing a two-dimensional XY array of pixel 
15 elements, 

(c) a pointer for indicating selected positions and orientations from which 
subsurface views are to be reconstmcted, and 

(d) a computational device operatively connected to the data file, pointer, 
and display screen for (i) determining the position and orientation of the pointer, (ii) 

20 defining a plurality of rays extending between a selected pointer position and each 
of a plurality of points in a two-dimensional XY array, where (iia) the an^ay points 
correspond to array pixels in the display screen, and (iib) at least some of the rays 
pass through a plurality of scan-date voxels in the patient target region, (iii) for 
each ray, for each ray, summing the M-bit values of the voxels along that ray, and 

25 (iv) displaying the image constructed of gray-scale values representing the 
summed M-bit density values at each pixel in said display screen. 

12. The system of claim 1 1 . for use in real-time evaluation of the displayed 
images, wherein the display screen contains a two-dimensional XY array of pixel 

30 elements, each element containing multiple N-bit registers for receiving digital- 
scale values for each of multiple colors, and said computational device perfomns 
the summing operation for each ray by (i) distributing the M bits of the voxel 
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density values among the multiple N-bit registers of the associated pixel, such that 
one or more bit positions of each M-bit value is assigned to a selected register of 
that pixel, and (ii) summing the M-bit values of the voxels along that ray by (iia) 
individually summing the one or more bit-position values in each associated pixel 
5 register, and (iib) determining the sum of the M-bit values along the associated ray 
from the values in the individual registers. 

1 3. A method of registering volumetric scan data of a patient surgical site 
with actual patient position, comprising 

10 (a) obtaining, at a scan station, volumetric scan data of a surgical site of a 

patient, including data relating to plural selected features in the region, and storing 
the scan data in digital fonm in a scan-data file, 

(b) after moving the patient from the scan station to a surgical station, 
positioning an x-ray source and a detector screen for imaging at least a portion of 

1 5 said target region fluoroscopically at a selected orientation, 

(c) determining the coordinates of said x-ray source and detector at said 
positions in a fixed coordinate system, 

(d) activating the x-ray source to produce a fluoroscopic image of the 
patient region on said detector, 

20 (e) determining the image coordinates of said selected features in the 

target region in said fluoroscopic image in the fixed coordinate system, 

(f) using the coordinates detemiined in (c) and (e) to detemnine the actual 
coordinates of the selected features in said fixed coordinate system, and 
(f) matching the coordinates determined in (f) with those of the same 
25 features in the scan data, to place the scan data in the frame of reference of the 
patient. 

14. The method of claim 13, which further includes using the scan data to 
construct a subsurface image of the patient, as viewed from a selected position 

30 and orientation in the surgical station. 
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15. The method of claim 14, wherein said selected position and orientation 
is respect to the tip of a surgical instmment, and which further includes tracking 
the position of said instrument in the surgical station. 

5 16. The method of claim 13, wherein said plural selected features include 

at least three features, and steps (b)-(e) are carried out at different selected 
positions of the x-ray source and detector screen. 

17. A system for registering volumetric scan data of a surgical site with 
10 actual patient position, comprising 

(a) a scan-date file for storing, in digital form, volumetric scan data of a 
surgical site of a patient, including data relating to plural selected features in the 
region, 

(b) an x-ray source and a detector screen for imaging at least a portion of 
1 5 said target region fluoroscopically at a selected orientation, 

(c) a fixed-position sensor for determining the coordinates of said x-ray 
source and detector at said positions in a fixed coordinate system, and 

(d) a computational device operatively connected to the data file, detector 
screen, and fixed-position sensor, for (i) determining the actual coordinates of 

20 selected features of the surgical site in said fixed coordinate system from the 
determined coordinates of such features on the detector screen and from the 
coordinates of said screen, and (ii) matching the actual coordinates of said 
features with those of the same features in the scan data, to place the scan data 
in the frame of reference of the patient. 
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